Hierarhical Models with Ternary Latent Variable
Youjin Lee
Models
Most of the structure of likelihood is same as that in hierarhical models with binary latent variable except that Bernoulli distributions now turn into Multinomial distributions. Models are based on [model 2] \(P(A, X, Z) = P( A | Z) P(Z | X) P(X)\).
- Generate iid node attribute with a parameter \(u = (u_{1}, u_{2})\)
\[X_{i} \overset{i.i.d}{\sim} Multinomial(u_{1}, u_{2}, 1 - u_{1} - u_{2} ), i = 1,... , n\]
Note that \(X\) should not be interpreted as a nominal, categorical variable since Euclidean distance was used to measure the distance of \(X\).
- Generate latent variables conditional on \(X\) as a function of \(\omega\)
\[\begin{align} Z_{i} | X_{i} & \overset{i.i.d}{\sim} Multinomial( 1 ; \pi_{1}(X_{i}), \pi_{2}(X_{i}), \pi_{3}(X_{i}) ) = Multinomial(1; \pi_{k} = (1/3 + \omega) I(X_{i} = k ) + (2/3 - \omega ) I(X_{i} \neq k) /2 , k = 1,2,3 ) \\ & \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinomial(1; \color{red}{\frac{1}{3} + \omega}, \frac{2/3 - \omega}{2} , \frac{2/3 - \omega}{2} ) & X_{i} = 1 \\ Multinomial(1; \frac{2/3 - \omega}{2}, \color{red}{\frac{1}{3} + \omega} , \frac{2/3 - \omega}{2} ) & X_{i} = 2 \\ Multinomial(1; \frac{2/3 - \omega}{2}, \frac{2/3 - \omega}{2}, \color{red}{\frac{1}{3} + \omega} ) & X_{i} = 3 \end{array} \right. \end{align}\]
- Generate local dependent graph : \(\phi = (p_{k}, q_{k,l}; k,l = 1,... , K, k < l)\)
\[\begin{align} P_{\phi}(A = a | Z = z) & = \prod\limits_{k} P_{\phi}(A_{ij} = a_{ij} | Z = z) \times \prod\limits_{k < l} P_{\phi} (Y_{kl} = y_{kl} | Z = z) \\ & = \prod\limits_{Z_{i} = Z_{j} = 1}^{K} {p_{k}}^{a_{ij}}(1-p_{k})^{a_{ij}} \prod\limits_{Z_{i} \neq Z_{j}} {q_{kl}}^{a_{ij}}(1 - q_{kl})^{a_{ij}} \end{align}\]
- Joint likelihood of \((A_{ij}, X_{i}, Z_{i} ; i,j=1,... ,n)\) under \(A \perp X \big| Z\) can be represented as :
\[P_{\theta}(A, X, Z) = P_{\phi}(A | Z) P_{\omega}(Z | X) P_{u}(X)\]
[sim] three.1
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} = Z_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1,... , n.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q12 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0 | 0.5 | 0.5 | 0.5 | 0.1 | 0.1 | 0.1 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.08 | 0.05 | 0.06 |
| local optimal | 0.11 | 0.14 | 0.13 |
[sim] three.2
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q23 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0.17 | 0.5 | 0.5 | 0.5 | 0.1 | 0.1 | 0.1 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.82 | 0.87 | 0.84 |
| local optimal | 0.93 | 0.96 | 0.92 |
[sim] three.3
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} 0.3 & 0.3 & 0.3 \\ \hline 0.3 & 0.3 & 0.3 \\ \hline 0.3 & 0.3 & 0.3 \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q23 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0.17 | 0.3 | 0.3 | 0.3 | 0.3 | 0.3 | 0.3 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.06 | 0.06 | 0.11 |
| local optimal | 0.09 | 0.16 | 0.14 |
[sim] three.4
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & \color{red}{0.5} & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q23 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0.17 | 0.5 | 0.5 | 0.1 | 0.1 | 0.1 | 0.1 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.89 | 0.73 | 0.75 |
| local optimal | 0.93 | 0.87 | 0.89 |
[sim] three.5
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q23 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0.17 | 0.5 | 0.1 | 0.1 | 0.1 | 0.1 | 0.1 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.57 | 0.66 | 0.62 |
| local optimal | 0.80 | 0.75 | 0.71 |
[sim] three.6
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{0.5} & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & 0.1 \\ \hline 0.1 & 0.1 & \color{red}{0.5} \end{array} \right]\]
| u1 | u2 | w | p1 | p2 | p3 | q12 | q13 | q23 |
|---|---|---|---|---|---|---|---|---|
| 0.33 | 0.33 | 0.17 | 0.5 | 0.1 | 0.5 | 0.1 | 0.1 | 0.1 |
| t=1 | t=5 | t=20 | |
|---|---|---|---|
| global test | 0.82 | 0.88 | 0.89 |
| local optimal | 0.98 | 0.96 | 0.95 |
Summary 1
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & q & q \\ \hline q & \color{red}{p} & q \\ \hline q & q & \color{red}{p} \end{array} \right]\]
Power heatmaps
- Global Power
- Optimal Power
Optimal power
Discrepancy between local and global scale
Diffusion Process
Summary 2
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & r & r \\ \hline r & \color{red}{p} & r \\ \hline r & r & r \end{array} \right]\]
Power heatmaps
- Global Power
- Optimal Power
Optimal power
Discrepancy between local and global scale
Diffusion Process
Summary 3
\[X_{i} \overset{i.i.d}{\sim} Multinorm(1/3, 1/3, 1/3), i = 1, ... ,n.\]
\[Z_{i} | X_{i} \overset{i.i.d}{\sim} \left\{ \begin{array}{cc} Multinorm(1/2, 1/4, 1/4) & X_{i} = 1 \\ Multinorm(1/4, 1/2, 1/4) & X_{i} = 2 \\ Multinorm(1/4, 1/4, 1/2) & X_{i} = 3 \end{array} \right.\]
\[A_{G} \sim Bern \left[ \begin{array}{c|c|c} \color{red}{p} & q & \color{blue}{r} \\ \hline q & \color{red}{p} & q \\ \hline \color{blue}{r} & q & \color{red}{p} \end{array} \right]\]
Since block 1 (\(Z_{i} = 1\)) and block 3 (\(Z_{i} = 3\)) are most different, if \(q < r,\) local scale is more likely to be better than the global scale. Thus I on purpose consider the case where \(q < r.\)
r = q + 0.1
Power heatmaps
- Global Power
- Optimal Power
Optimal power
Discrepancy between local and global scale
Diffusion Process
r = q + 0.2
Power heatmaps
- Global Power
- Optimal Power
Optimal power
Discrepancy between local and global scale
Diffusion Process
